Learning Prototypical Event Structure from Photo Albums
نویسندگان
چکیده
Activities and events in our lives are structural, be it a vacation, a camping trip, or a wedding. While individual details vary, there are characteristic patterns that are specific to each of these scenarios. For example, a wedding typically consists of a sequence of events such as walking down the aisle, exchanging vows, and dancing. In this paper, we present a data-driven approach to learning event knowledge from a large collection of photo albums. We formulate the task as constrained optimization to induce the prototypical temporal structure of an event, integrating both visual and textual cues. Comprehensive evaluation demonstrates that it is possible to learn multimodal knowledge of event structure from noisy web content.
منابع مشابه
Recognizing and Curating Photo Albums via Event - Specific Image Importance ( Supplementary Material )
In Figure 1 we show an example of the need to collect the multi-label ML-CUFED dataset with because of albums with ambiguous or multiple event types. The two albums in Figure 1 are both labeled as birthday events in CUFED, but they can also fall into the category of casual family/friends gathering. These two event types are not mutually exclusive. Moreover, intuitively, we would consider the al...
متن کاملIntelligent photo clustering with user interaction and distance metric learning
Photo clustering is an effective way to organize albums and it is useful in many applications, such as photo browsing and tagging. But automatic photo clustering is not an easy task due to the large variation of photo content. In this paper, we propose an interactive photo clustering paradigm that jointly explores human and computer. In this paradigm, the photo clustering task is semi-automatic...
متن کاملLearning Visual Storylines with Skipping Recurrent Neural Networks
What does a typical visit to Paris look like? Do people first take photos of the Louvre and then the Eiffel Tower? Can we visually model a temporal event like “Paris Vacation” using current frameworks? In this paper, we explore how we can automatically learn the temporal aspects, or storylines of visual concepts from web data. Previous attempts focus on consecutive image-to-image transitions an...
متن کاملRecognizing and Curating Photo Albums via Event-Specific Image Importance
Automatic organization of personal photos is a problem with many real world applications, and can be divided into two main tasks: recognizing the event type of the photo collection, and selecting interesting images from the collection. In this paper, we attempt to simultaneously solve both tasks: album-wise event recognition and imagewise importance prediction. We collected an album dataset wit...
متن کاملTuneSensor: A Semantic-Driven Music Recommendation Service For Digital Photo Albums
Digital photo album softwares like iPhoto have enjoyed great popularity for years. These years, online photo album services (e.g., Flickr and Picasa) have been becoming more and more popular with the development of social Web. In this paper, we demonstrate our effort called TuneSensor to recommend music for photo albums automatically. In particular, we exploit semantic data to represent both im...
متن کامل